-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce fail-fast feature for dynamic test execution #838
Conversation
Convert Scenario annotation to blocking flag in DynamicNode.
Removed all `Stream.of(...)` usages from the demo: returning an array of `DynamicNode` was already supported.
If you end up not using merging this, there is a small side project I could recommend. 😉 |
Please note that the "blocking" feature you implemented in this PR is directly related to #431. |
I think there needs to be a way to specify a blocking container named something like @TestFactory
@DisplayName("Web Security Scenario")
Stream<DynamicNode> scenarioTests() {
return Stream.of(
dynamicTest("Visit page requiring authorization while not logged in", () -> {
// attempt to visit page which requires that a user is logged in
// assert user is redirected to login page
}),
dynamicScenario("Log-in", Stream.of(
dynamicStep("Log-in button is disabled", () -> {
// assert user cannot submit login form without entering credentials
}),
dynamicStep("Log-in button becomes enabled after entering credentials", () -> {
// assert log-in button is enabled once credentials are entered
}),
dynamicStep("Visit third page requiring authorization while logged in", () -> {
// submit login form with valid credentials
// assert user is redirected back to previous page requiring authorization
// fail("you shall not pass");
})
)),
dynamicContainer("Can access several pages while logged in", Stream.of(
dynamicTest("Visit second page requiring authorization while logged in", () -> {
// visit another page which requires that a user is logged in
// assert user can access page
}),
dynamicTest("Visit third page requiring authorization while logged in", () -> {
// visit another page which requires that a user is logged in
// assert user can access page
})
)),
dynamicTest("Log-out", () -> {
// visit logout URL
// assert user has been logged out
})
);
} |
Description updated. @mfulton26 Does the new required strategy meet your needs? |
I believe so. Thank you. |
replace "dynamic" and leave "Test"
leave "dynamic" and replace "Test"
|
Another alternative, a separate method called bool shouldContinue()
It is called after each step is executed.
Perhaps the quality of any of these names would easier to judge of there
was a sample code fragment.
Cheers
Mark
…On May 7, 2017 1:11 PM, "Christian Stein" ***@***.***> wrote:
DynamicTest.requiredTest(...) is not the best name. What other name could
we use to describe *"a dynamic test that has the power to halt the
execution loop of the @TestFactory-annotated method"*?
replace "dynamic" and leave "Test"
- DynamicTest.requiredTest
- DynamicTest.essentialTest
- DynamicTest.gatewayTest
leave "dynamic" and replace "Test"
- DynamicTest.dynamicCheckPoint
- DynamicTest.dynamicGuard
- DynamicTest.dynamicBarrier
- DynamicTest.dynamicStep
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#838 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAImZl2oWuoCoGFDIXOkorRiGoPmZrzJks5r3fsigaJpZM4NRnqa>
.
|
Mh. How do you pass the method (or any other callback handle) to the execution cycle? Can this "registerPredicateEvaluatedAfterEachDynamicTest(this::shouldContinue);" called more than once?
Look at the description text of this issue; it uses |
@christian - working from my phone is hazardous. Trying to do it while kids
basketball skills session is in progress is worse.
My solution is clearly not fantastic either. My mental challenge, even
after reviewing the code, this whole thing feels awkward semantically. Your
names for Guard and Barrier seem best, but it feels all very special case.
Compared to rest of the JUnit 5 (and even JUnit 4) model this feels very
different. I can't yet see a better approach, I just feel that I'm not
seeing right now.
Perhaps with the existing implementation someone could show us a sample
test case - that illustrated the underlying problem.
Time to drive kids home
Cheers
Mark
…On Sun, May 7, 2017 at 3:43 PM, Christian Stein ***@***.***> wrote:
[...] It is called after each step is executed.
Mh. How do you pass the method (or any other callback handle) to the
execution cycle? Can this "registerPredicateEvaluatedAfte
rEachDynamicTest(this::shouldContinue);" called more than once?
Perhaps the quality of any of these names would easier to judge of there
was a sample code fragment.
Look at the description text of this issue; it uses requiredTest.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#838 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAImZtB1c3nMWmY0ZpSW9HhmjfkV-PtLks5r3h7MgaJpZM4NRnqa>
.
--
[image: headshot-square-300x300]
<http://www.flickr.com/photos/36331075@N00/9674877046/> *Mark Levison* | 1
(877) 248-8277 | Twitter <https://twitter.com/mlevison> | LinkedIn
<http://ca.linkedin.com/in/marklevison> | Facebook
<https://www.facebook.com/agilepainrelief>
Certified ScrumMaster Training: Vancouver
<http://agilepainrelief.com/courses/vancouver> | Edmonton
<http://agilepainrelief.com/courses/edmonton> | Ottawa
<http://agilepainrelief.com/courses/ottawa> | Montreal
<http://agilepainrelief.com/courses/montreal> | Toronto
<http://agilepainrelief.com/courses/toronto>
Certified Product Owner & Private Training also available ~ Our Training
Schedule <http://agilepainrelief.com/courses/certified-scrum-agile-training>
Agile Pain Relief Consulting <http://agilepainrelief.com/> | Notes from a
Tool User <http://agilepainrelief.com/notesfromatooluser>
Proud Sponsor of Agile Tour Gatineau Ottawa <http://goagiletour.ca/> and Agile
Coach Camp Canada <http://agilecoachcampcanada.wordpress.com/>
|
What about adding an optional parameter to |
Christian - to my limited tastes in this area is an improvement. I like the
fact that it gets rid of something special which feels more inline with
JUnit. I've never loved outparams, but at least all dynamicTest() methods
are now the same. I also like that any test could be reason to terminate
the sequence for any reason. Slightly better would more expressive enum.
I'm guessing there are three states: continue; skip; halt. Skip might be
used when the test should be skipped because a pre-condition isn't
currently valid. Again lacking a concrete and specific example I'm
struggling to make my own thinking clear.
Cheers
Mark
…On Mon, May 8, 2017 at 3:47 AM, Christian Stein ***@***.***> wrote:
What about adding an optional parameter to DynamicTest.dynamicTest(String,
Executable, Predicate) is evaluated after the executable ran. If it
evaluates to true the dynamic test execution proceeds, or otherwise
terminates.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#838 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAImZjSoo5Zq9SGwS8rMsNs1JK0TyiKeks5r3siggaJpZM4NRnqa>
.
--
[image: headshot-square-300x300]
<http://www.flickr.com/photos/36331075@N00/9674877046/> *Mark Levison* | 1
(877) 248-8277 | Twitter <https://twitter.com/mlevison> | LinkedIn
<http://ca.linkedin.com/in/marklevison> | Facebook
<https://www.facebook.com/agilepainrelief>
Certified ScrumMaster Training: Vancouver
<http://agilepainrelief.com/courses/vancouver> | Edmonton
<http://agilepainrelief.com/courses/edmonton> | Ottawa
<http://agilepainrelief.com/courses/ottawa> | Montreal
<http://agilepainrelief.com/courses/montreal> | Toronto
<http://agilepainrelief.com/courses/toronto>
Certified Product Owner & Private Training also available ~ Our Training
Schedule <http://agilepainrelief.com/courses/certified-scrum-agile-training>
Agile Pain Relief Consulting <http://agilepainrelief.com/> | Notes from a
Tool User <http://agilepainrelief.com/notesfromatooluser>
Proud Sponsor of Agile Tour Gatineau Ottawa <http://goagiletour.ca/> and Agile
Coach Camp Canada <http://agilecoachcampcanada.wordpress.com/>
|
Closing this PR with comment: "Back to the design board." Follow-up at #431 |
Overview
This PR provides tools around the dynamic tests feature to accomplish a scenario-like way of expressing related tests. The new factory method
DynamicTest.requiredTest(String displayName, Executable executable)
creates dynamic test instances, that require to finish successfully. If the dynamic test execution loop detects an unsuccessful required test, it stops the loop immediatly. This can happen at any dynamic container level. If the execution loop is halted, no further dynamic test is registered and therefore not executed.Example usage
Copied from DynamicTestsDemo.java with Log-in and third page visit being required dynamic test. They require to finish with a successful
TestExecutionResult
Related issues
Deliverables
I hereby agree to the terms of the JUnit Contributor License Agreement.
Definition of Done
@API
annotations